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ABSTRACT 

This report presents findings from Beta, one of four 
school districts examined in a series of case studies that 
investigated the complex relationship between reading assessment and 
instruction. The research was situated in the context of school 
decision meking. Teachers, principals, parents, students, and central 
office staff were interviewed to determine how decisions were made in 
the district and how that decision-making process influenced 
assessment and instruction. In addition, teachers were observed, and 
discussions were conducted with them about the observations. A 
constant comparative method was used to identify patterns in the 
data. Findings showed that standardized-test data played a powerful 
role in Beta, with teachers, principals, and central office staff 
reporting that such data were the most viable way to understand how 
students were doing. Teachers* assessments were rarely mentioned. 
Within the classroom, teachers felt that they needed to "cover" the 
textbooks before state and district testing so that their students 
would score well. Teachers spent classroom time helping students 
prepare for the test; indeed, sometimes the curriculum for the entire 
year was driven by the test. Curriculum was synonymous with 
textbooks, and using textbooks was expected to lead to good test 
scores. Educators in Beta seemed to feel that the tests measured what 
mattered, and that therefore it was reasonable to modify curriculum 
and instruction to match the test. (One table and three figures of 
data are included; a letter to schools describing the research 
project, and a list of interview and observation codes are attached.) 
(Author/RS) 
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Abstract 

This report presents findings from Beta, one of four school districts examined in a series of case studies 
that investigated the complex relationship between assessment and instruction. The research was 
situated in the context of school decision making. Teachers, principals, parents, students, and central 
office staff were interviewed to determine how decisions were made in the district and how that 
decision-making process influenced assessment and instruction. In addition, teachers were observed and 
discussions were conducted with them about the observations. The interviews were taperecorded and 
transcribed, and field notes from the observations were elaborated. A constant comparative method was 
used to identify patterns in the data. Findings showed that standardized-test data played a powerful role 
in Beta, with teachers, principals, and central office staff reporting that such data were the most viable 
way to understand how students were doing. Teachers' assessments were rarely mentioned. Within the 
classroom, teachers felt that they needed to "cover" the textbooks before state and district testing so that 
their students would score well Teachers spent classroom time helping students prepare for the test; 
indeed, sometimes the curriculum for the entire year was driven by the test. Curriculum was 
synonymous with textbooks, and using textbooks was expected to lead to good test scores. Educators 
in Beta seemed to feel that the tests measured what mattered. They felt, therefore, that it was 
reasonable to modify curriculum to match the test. Indeed, specific standardized- test data were used 
to modify instruction (e.g., district-wide low scores on a particular item or set of items would result in 
requests for teachers to emphasize that particular area more). 
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ASSESSMENT AND DECISION MAKING IN BETA 



The field of reading education generaily seems to agree that assessment (defined as testing) drives 
instruction-that is, teachers teach to the test. A review of the literature suggests that this view of 
assessment is grounded, for the most part, in large-scale studies that have found that teaching practices 
change in response to pressures to increase student performance or to help them "do well" on 
standardized, norm-referenced tests (see, e.g., Herman & Dorr-Bremme, 1983; Popham et al. 1985; 
Salmon-Cox, 1981). 

Those of us involved in this research project wondered what these findings meant in terms of particular 
teachers in particular classrooms in particular districts. Our wondering led us to formulate a number 
of specific questions: What was life like in a school that was attempting to raise its test scores? Were 
daily patterns of instruction in those schools different from those in schools that did not seeoi highly 
invested in raising test scores? What was the relationship between textbook orders (kind and company) 
to testing? Might an individual teacher feel unaffected by test pressures, yet be required to use 
materials that had been specifically chosen to match test items? And what about policies for passing 
versus retaining students? Might a teacher feel relatively free from test pressures during the year, then 
be told that only students with certain reading levels could pass to the next grade, a grade in which 
standardized tests were administered? 

To address these questions, and thus to move our understanding from the abstract (research says that 
testing drives instruction) to the concrete (what does this mean in the lives of particular 
teachers/schools/districts?), we conducted case-study research in four school districts. 1 

For our research, we chose districts we thought would have different ideas about the relationship 
between assessment and instruction: (a) a district with •<> reputation for being a low stakes district-scores 
were acknowledged and then filed; (b) a high stakes or test-driven district; (c) a district known for its 
high test scores and the belief of lis personnel that the consistent pattern of such scores gave them 
license to do pretty much whatever they wanted (although the district felt that the community would 
"pull in the reins" if test scores dropped); and (d) a district concerned about its test scores because of 
how they were perceived in the community. Personnel in this district worried that what they considered 
"low" test scores meant that their reading program needed to be changed. We designated these districts 
Alpha, Beta, Gamma, and Delta. 

We chose and contacted Alpha, Beta, and Gamma, explaining our interest to central office personnel 
and asking if their district would be willing to participate in the study. Meanwhile, the superintendent 
of the fourth district, Delta, contacted us and asked to participate in the study, 

In our conversations with school personnel, we explained (see letter in Appendix A) that our interest 
was in the relationship between standardized tests and instruction, and that we wanted to situate both 
tests and instruction within a broader framework of instructional decision making so that we could better 
understand the more subtle influences of one on the other (e.g., textbook purchasing policies). Wc also 
explained that wc were interested in the seldom talked about assessment that was not test (e.g., teacher 
observation) and the relationship of that form of assessment to instruction. All participants, therefore, 
understood that we were interested in decision making as it related to assessment (both as test and 
not-as-test) and instruction. 



l The four case studies are available as Center for the Study of Reading Technical Reports. 
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Before the study began, and based on time and staff limitations, we decided to focus on two buildings 
per district, two teachers per building. The districts responded differently to our plan. In Alpha, central 
office staff notified all teachers that we wanted to conduct a study and asked them to contact us if they 
were interested. In that district, 7 teachers in one building and 2 in another participated. In Beta, 
central office staff decided which buildings and teachers would participate. In Gamma and Delta, 
central office staff invited teachers and principals to a meeting to hear about the study and then choose 
2 schools from among those interested. 

We used interviews and observations as our means of collecting data, interviewing key participants in 
all four sites: central office personnel, principals, teachers, parents, and students. The interviews with 
teachers were tied to our observations of their classrooms. The first interview was held prior to the first 
observation, and conversation-like interviews were held after each of 3 observations. The fifth interview 
followed the final observation. With the participants' permission, the interviews were tape-recorded and 
transcribed. Field notes were taken during the observations and elaborated afterward. These field notes 
-were returned to the participants for their comments and, when appropriate, further elaboration. 

The observations provided an opportunity to ground our interview questions in the. concreteness of 
teachers* personal experiences. We could see which books they used, how they graded papers, and how 
they responded to students. We were then able to ask teachers about the relationship of those 
classroom-based decisions to the broader issues of assessment and instruction. 

The data (audiotapes and elaborated field notes) were read and coded, using descriptive codes (see 
Appendix B). Tenuous labels emerged from the initial reading of the data. These codes were revised 
and refined until the codes adequately captured the content of what we had seen and heard. The codes 
were intended to facilitate analysis. The process of careful, descriptive coding also focused our reading 
of the data and helped us begin to identify patterns in the data. 

After the coding was completed, one member of the research team took primary responsibility for each 
district. A constant comparative approach was used in the analysis. Each researcher read and reread 
the data, looking for and identifying patterns. Once patterns had been identified, the data were read 
at least one more time for evidence that might disconfirm the patterns. The researcher then presented 
the patterns in a case study that aptly captured what we had learned about assessment and instruction 
in that district. 2 Meanwhile, members of the research team continued to meet with each other, sharing 
possibilities and patterns. These case studies were returned to all participants for their feedback and 
changes, as necessary, were made in subsequent versions of the case studies. 

Once we had a clear understanding of the patterns within districts, the data were read again. This time 
a new member of the research team, building conceptually on the analysis done in each district, 
reanalyzed all the data to identify patterns across districts. The data were then reread to confirm those 
patterns and identify salient examples, and read a final time to find negative cases, instances in which 
the pattern could not be confirmed. Once all patterns were confirmed, and therefore considered 
trustworthy, a cross-site analysis was prepared 3 . 

The District in Context 

Beta School District is in a midwestern city of about 59,000 people. The city contains a large university, 
a community college, a few manufacturers and a number of service industries. Several members ol the 



2 A fifth case study was conducted in Ohio. It is also available as a Technical Report. 
■^The cross-site analysis is also avail able as a Technical Report. 
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research team were familiar with this district and thought that Beta might offer a different perspect. e 
on tn re Snship between assessment and instruction than the perspective we ant>cipated finding m 
Aloha SoScaUy their experiences led them to speculate that Beta might be more top-down 
Jd£i«2S3SS that differences in the decision-making model might affect the relationsh.p between 
assessment and instruction. 

David Pearson contacted an Assistant Superintendent, Mr. Sawyer, to ask if the district would be willing 
£ «dy. Mr. Sawyer seemed interested and scheduled a meeting time to talk more 
abo P uTthe P smdy. He inviLd the prhcipals of two demographically similar schools, Beta I and Beta II, 
to attend that meeting (see Table 1). 

[Insert Table 1 about here.] 

Mr Dixon and Mr. Sandstrom, principals of Beta I and Beta II, decided which teachers in their schools 
would r t SUe and arranged a tentative time for the research team and the teachers to meet. The 
ZlLTb See tte teachefs and researchers took place a few weeks later in one : of the ■ Partieipatmg 
"hods Te researchers explained the study, answered questions and scheduled the first round of 
observations. 

Observations and interviews were conducted over a period of three months in the spring Four -teachers 
^oTom each school, participated in the study, and a researcher was ass.gned^ to , eac teach. O 
marchers interviewed each principal. At the request of Mr. Sawyer, all central once 
aSra^ DT Montgomery, the Superintendent; Mr. Sawyer, Assistant Superintendent of 
ctrSum and Instruction; Ms Warner, Assistant Superintendent of Support Services; Mr. Hams, 
S of ^tmeTary Curriculum; and Ms. Cooper, Director of Secondary Curriculum-were 
interviewed together. 

The follow fall we began to analyze the data we had collected. Our goal was to understand literacy 
tofioHfea h classroom and to understand that instruction in the context of assessment and 
SCm^ing across the district. We used the data from teacher interviews and observations to 
t epare tofionaT'gnettes for each classroom to capture what was unique to each teacher. The 
Sues Tobserved Trz presented here to provide a context for what we learned about decs.on 
making in Beta. 

Literacy Instruction 

Beta I Obsen'ations 

Ret a I school was built in the early 1960s and is a one-story structure with classrooms on the outside 
I is The inter or o he school building consists of a garden courtyard, the gymnasium/cafeteria, and 
The tealheXn^e. Painted murals and large bulletin boards with children's art work and papers^ 
on the walls in the hall. In Beta I School, we observed and interviewed Ms. Holmes, a first grade 
teacher, and Ms. Carter, a third-grade teacher. 

Ms. Holmes, First Grade 

Ms Holmes' literacy instruction followed the district-selected textbook. Her instruction emphasized the 
SttSde children needed to become M^Z 
tradebooks was important for her first graders; she read these books to the chddren, hnked basal stones 
to tradebooks, and provided tradebooks for students to read mdcpcndcnt.ly. 
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On the days we observed, Ms. Holmes began her morning instruction by introducing worksheets that 
the children would do independently at their seats. The class was given an English textbook worksheet 
on the correct usage of is and are, math worksheets, a story with sentences to cut and paste into the 
correct sequence, and a dot-to-dot bunny coloring sheet. After these worksheets had been introduced, 
she told them, "When your work is done, you may read a book or write a story." 

When the children were settled working independently, Ms. Holmes began to work with the three 
reading groups. During this time, Ms. Holmes followed a pattern similar to the basal manual lessons. 
She began the lesson by preparing the students for the story in their basal reader. In one reading group, 
she showed students the tradebook from which the basal story and lesson were taken. During another 
reading group, Ms. Holmes brought a tradebook that had the same poem the children were to read in 
the basal. She asked them to compare the pictures and to tell her other pieces that they had read by 
the same author. After reading the poem to the children, she asked them questions about the poem 
and incorporated discussions about other books she had read to them. She said she wanted the children 
to be aware that the basal stories and poems were available in the library, and she encouraged them to 
check out the tradebook that the basal series adapted for instruction. As a preparation for reading the 
story, she discussed the topic of the basal selection to make it familiar to the students and she 
introduced vocabulary that she felt they would find difficult. The vocabulary she taught was written in 
sentences which the children took turns reading. Ms. Holmes used this time to reinforce or teach 
phonics or other skills. 

In Writing to Read we learned aw says /aw/ as in saw. In our reading book it also tells 
us that au says /aw/. 

Ms. Holmes, 3/29/89 

After skills and vocabulary were taught and the story was introduced, Ms. Holmes focused on reading 
the story. She gave the students a purpose for reading (e.g., "Read these two pages to find out where 
hen lays her eggs and why she lays them where she lays them."). The children read those pages silently, 
and Ms. Holmes asked questions about what they read. 

Ms. Holmes: What happens to the nest? What does she put there? What do we call 
that when eggs crack open? 

Martin: Chicks 

Lucy: Hatch 

Field notes, Ms. Holmes classroom, 3/29/89 

This pattern was repeated until the story was completed, and then Ms. Holmes assigned basal workbook 
pages. She reported that if the children already knew a skill to be practiced in the workbook or if the 
page was too easy, she would skip those pages. She also would work with individual students to keep 
them current in the basal and their workbooks. 

It's based on what they've had trouble with previously. I really like keeping everyone 
caught up. I don't like for anyone to feel like they're behind or not good at something. 

Ms. Holmes, 3/30/89 

Ms. Holmes used informal assessment to determine whether students needed instruction for skills not 
taught in the basal program. During one observation, for example, she used informal assessment 
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techniques to learn about a new girl in the class. She asked the girl questions concerning specific skills 
such as knowing the /al/ sound. Ms. Holmes told us that she felt it was important to determine skills 
her children did not know so she could make instructional decisions. 

Ms. Carter, Third Grade 

During our third observation of her classroom, Ms. Carter conducted a whole-class lesson on the book, 
Annie and the Old One (Miska, 1971), which had appeared as a story in a basal reader. Because she 
had completed the required basals for her reading groups, she chose to have the whole class read the 
story. The pattern of discussion observed during this lesson matched those we observed in reading 
groups during the first and second observations. 

Ms. Carter began the lesson by going over the vocabulary. She provided the students with a worksheet 
of sentences she had written using the vocabulary she had decided was difficult. As she noted, 

I read the story before hand and I pick any phrases and words I think they might not 
understand, and we talk about them. 

Ms. Carter, 3/3/89 

Some of the content sentences were: 

1. ) There was a low mesa near her own snug hogan. 

2. ) The tassels on the corn were turning brown. 

3. ) Annie sat beside a loom. 

Field notes, Ms. Carter's Classroom, 5/10/89 

After reading the sentences silently, the students were asked to read them aloud. For a few of the 
sentences, they had discussions, about the word to expand their understanding. 

Ms. Carter: What do they do on a loom, Sara? 

Sara says that she does not remember. 

Ms. Carter: Linda, can you help her out? 

Linda says that you weave on the loom. 

Ms. Carter: Right. And here's a picture- Annie and her mother are weaving. 

Field notes, Ms. Carter Classroom, 5/10/89 
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She continued the preparation for reading by telling them about the Navajos and showing where they 
lived by pointing out states on the map. The next step was to focus their attention on the story: 

While we are reading the first part, we want to find out why Annie doesn't want her 
grandmother to finish weaving the rug. 

Field notes, Ms. Carter's Classroom, 5/10/89 

Ms. Carter then read the title and asked, "What can we find out?" Various children gave suggestions, 
and she responded to their different answers. One child told her that they could find out who^was the 
old one. Another child said they could find out who Annie was. The third child stated that they could 
find out what the problem was. Ms. Carter then gave them a purpose for reading a page such as to find 
out what kind of chores Annie did to help her family. After reading a page she asked questions such 
as "What did the grandmother tell Annie on this page? Who can find that sentence?" The reading of 
the story generally followed this pattern: Ms. Carter set the purpose for reading a page or two, the class 
read silently, Ms. Carter asked questions about the page. The following was an example of this pattern: 

Ms. Carter: Read 312 and 313 to find out what Grandma announces and why they all 
got to choose something. 

The children read silently and Ms. Carter walks around the room. 

Ms. Carter: Who can find the sentence that tells what Grandma announces? This 
time I want you to find the sentence. What important things does the old one, does 
the grandmother say? 

She called on a student who read the sentence that when the grandmother is done 
weaving, she will leave mother earth. 

Another student whispers: She's gonna die. 

Ms. Carter: What does that mean? 

The student says that the grandmother is going to die .ind asks: How does she know? 

Ms. Carter: That's what we'll find out. Now what does, go back, find the sentence 
that tells what the father chooses. 

Field notes, Ms. Carter's Classroom, 5/10/89 

Ms. Carter explained that she used questions about the basal stories to determine vhether a student was 
ready for the next reader or a higher reading group. She also used questions to determine 
comprehension of the stories. She reported that she looked for understanding of the main idea. With 
difficult stories she had the students read page by page, checking on comprehension for each page. 

After reading the story page by page and answering Ms. Carter's questions, the students summarized 
the first part of the story. 

Ms. Carter: Now, what is the problem in the story? What happens? What is kind of 
sad? 

She called on a child who said that the grandmother was going to die, and that Annie 
didn't want her to. 
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Ms. Carter: When does the Grandmother say she is going to die, Margaret? 
Margaret: When she finishes the rug. 

Ms. Carter: Okay. In the second part we are going to find out what Annie learns. 

Field notes, Ms. Carter Classroom, 5/10/89 

The children finished reading the first part of the story and were given a worksheet with questions very 
much like those Ms. Carter had asked during the guided reading. The assignment was to write out 
answers to those questions. Some of those were: 

1. How is living on an Indian reservation different from living in a city. How is 
your life different from Annie's? 

2. What chores did Anne do to help her family? What chores do you do? 

3. Why do you think Annie wanted her mother to stop working on the rug? 

Field notes, Ms. Carter Classroom, 5/10/89 

As the children wrote answers, } *s. Carter walked around the room checking how the students were 
doing. Our observation ended 1 /hen it was time for the students to go to music. 

Beta II School Observations 

Beta II School was built in the later 1950s and is also a one-story building with self-contained 
classrooms. In the entryway of the school were two large murals-drawn by children. A wall in the main 
hallway had the names of every child in the school and rows of stars behind the names. This chart was 
part of a school- wide system of public recognition for attendance and good behavior. Framed pictures 
made by the children hung in the hallways, and bulletin boards crowded with children's writing and art 
work marked the entry to each classroom. In the classrooms, the school-wide rules for classroom 
behavior were displayed prominently on bulletin boards. Ms. Eaton, a second-grade teacher, and Ms. 
Boyd, a third-grade teacher, participated in the Beta II study. 

Ms. Eaton, Second Grade 

Ms. Eaton had previously taught in an out-of-state school district that used a literature-based program 
for reading instruction. When she took the job at Beta II, Ms. Eaton was aware that she' would have 
to use the direct instruction approach to reading adopted by the school. While Ms. Eaton did use the 
materials and did follow the school's reading approach, she felt limited by it. Therefore, she 
supplemented that program with a variety of integrated language arts activities, including a writing 
component, that she felt were important for her students. These supplementary activities fit her 
personal philosophy. 

Part of my day, the decision has pretty much been made lor me. The other part of my 
day, I feel, it's my philosophical obligation to do what I feel is best for the children, 
within a certain framework provided by the district. I mean I have a second-grade 
curriculum guide that tells me the units I should be leaching. . . . How I accomplish 
these objectives is my decision and challenge. 

Ms. Eaton, 3/9/89 

In our second observation, Ms. Eaton began the morning by going over the activities the children would 
do that day. These activities were listed on the chalkboard: 
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House page 

Crossword puzzle (on animal homes) 
Bunny page 

Jellybean Math Worksheet (color/count) 
Bunny basket 

Field notes, Ms. Eaton's Classroom, 3/15/89 

Ms. Eaton offered some suggestions to help children choose which activity to do when. They were given 
the freedom to decide when and how to complete the four activities, but they were expected to complete 
all four by the end of the day. 

After going over the activities for the day, Ms. Eaton was ready to begin instruction. She used this 
transition time as an opportunity to prepare them for the next activity. 

Ms. Eaton: Everybody put away your markers and scissors. I'm going to be thinking- 
markers and scissors are frozen on your desk tops—I'm going to be thinking about 
something that has two legs, a top like a rectangle, it opens and closes, and you can 
think of it as a house for books. Lily? 

Lily: A desk 

Ms. Eaton: I'm thinking about an object that could be a house for something. I'm 
thinking of something that is also a rectangle, something that has 24 crayons, or 64 
crayons. . . . Lily? 

Lily: A crayon box 

Field notes, Ms. Eaton Classroom, 3/15/89 

When the children were settled, she told them she was going to read the book, A House is a House for 
Me by Mary Ann Hoberman (1978). She showed the book and thumbed through it making comments 
about the illustrations. She then read the book, asking questions and making brief comments as she 
read. When she read about beds as a home for bedbugs, the students commented: 

Joe: I never had one 

Mike: They're there though, you just can't always see 'em! 

Ms. Eaton: Let's think of more houses. 

Field notes, Ms. Eaton's Classroom, 3/15/89 

She continued to read and the children chimed in on the phrase "but a house is a house for me." When 
she finished reading, she asked the children for their own ideas. She pointed to the poster behind her 

which read: "A is a house for a but a house is a house for me." The children gave 

many ideas for the blanks and she wrote them all down. Some of them were as follows: 

Brad: A cat is a home for a mouse. 

Ms. Eaton looked puzzled. 
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Mallory: Yeah, if he swallowed it! 

Ms. Eaton: Yes, if a cat swallowed the mouse, it would be a house for it. 
Susie: A brain is a house for ideas. 
Ms. Eaton: Good thinking, Susie. 
John: A head is a house for your brain. 

Ms. Eaton: A head is a house for your brain, or you could use the word skull instead 
of head. 

Field notes, Ms. Eaton's Classroom, 3/15/89 

Although she made this suggestion, she wrote down the words John gave her. She consistently used the 
language that the children used. 

After she had written down many ideas on the poster, she handed out dittos with the same sentences 
(with blank spaces) as on the poster. These dittos were then made into a class book. The children had 
the option to make their own book. 

Henry: Sky is a house for clouds. Ear is home for earwax. An ear is a home for a 
earring, unless you put it in your nose. 

Ms. Eaton: Good ideas, Henry. 

She added Henry's ideas to her list. 

Ms. Eaton: Eyes up here, look at the ideas we have. Now I want you to save the rest 
of your ideas. You can write them on this page. [She held up the ditto.] If you have 
more than one idea you may put them in one of these books. 

Field notes, Ms. Eaton's Classroom, 3/15/89 

Ms. Boyd, Third Grade 

Ms. Boyd's classroom was bright and colorful. Fish made of newspapers hung from the ceilings, plants 
were placed throughout the room, and posters of animals and plants hung on the walls. Posters also 
hung on the doors of the closets: Classroom Helpers, Citizen of the Week, School Rules, Classroom 
Rules, Playground Rules, Luncheon Rules, Substance Abuse, and Steps for Creative Writing. 

All third-grade students had been tested with a reading series (Distar) placement test and then divided 
into six groups. 

That is the nice thing about the direct instruction. It is not all of this guessing game 
about, "what do I do with the children, where do I place them." It's not a guessing 
game. 

Ms. Boyd, 3/15/89 

Ms. Boyd taught two of these reading t roups, and the other third-grade teachers taught the other four 
reading groups. She used the Distar reading scries in each group for 45 minutes every day. She also 
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noted that each lesson in the series took two days to teach. On the first day she taught the vocabulary 
and guided the students through an initial reading of the story. On the second day the students reread 
the story, completed a related workbook page and, if there was time, worked on their independent 
writing. 

During two of our observations, Ms. Boyd passed out worksheets to the whole class before meeting with 
the reading groups. As the children read aloud the words listed on one worksheet, Ms. Boyd used the 
technique of clapping once to indicate when the children should say the next word. 

Students: Cow. Mexico. Happy. 

Ms. Boyd: You forgot a word. 

Students: Canada. 

Ms. Boyd: Let's start again. 

She claps and the students begin with the first word. 
Ms. Boyd: Next. 

She claps and the students repeat the list of words. 

Field notes, Ms. Boyd's Classroom, 3/10/89 

During one of the reading group sessions, the children were reading about a fly named Herman who 
flew to different countries. Ms. Boyd followed the manual and the techniques of the series. The 
following excerpt illustrates how Ms. Boyd conducted her reading group when following the direct 
instruction manual. 

Ryan read from the reading book and hesitated. 

Ms. Boyd: The word is and. What's the word? 

Ryan: And 

Ryan continued to read, then Julie read aloud softly. 
Ms. Boyd: The word is come. What's the word? 
Julie: Come back 

Ms. Boyd: All right. Start again from the beginning. 
Julie read from the beginning. 

Ms. Boyd: Everybody, spell the word. The word is flies. 

The students spelled the word as Ms. Boyd snapped her fingers for each letter. Then 
George read. 

Ms. Boyd: What kinds of things do Hies do? 
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George: Eat food. 

Ryan: They like yellow things. 

Colleen: Buzz around. 

The children continued to read the story aloud. 

Field notes, Ms. Boyd's Classroom, 3/10/89 

Ms. Boyd: How was Charlotte's Web like the story? 
Henry: When they both died at the end. 
Ms. Boyd: Does a fly look exactly like its mother? 
Sally: Yes. 

Ms. Boyd: Does a fly grow? 

Andy: No, but on the inside it grows. 

Ms. Boyd: Let's compare Charlotte and Herman. 

Jake: They. . . 

Ms. Boyd: Raise your hand when you're ready. George, compare them. 
George: They're both insects. Wait. 
Ms. Boyd: Julie? 

Julie: They both have babies, but Charlotte is a spider, but Herman is a fly. 

Field notes, Ms. Boyd's Classroom, 3/10/89 
™^T&^*~* ^ «** ^en asfced about the 
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All four of the Beta teachers had reported that while they used mandated materials, how they taught 
the curriculum was their prerogative. Our observations supported their statements. We next integrated 
our understanding of literacy instruction at the classroom level with the information we had obtained 
from district administrators, principals, and students in order to determine how decisions were made 
in the district. As a result, we came to understand how these individuals viewed Beta's organizational 
structure, curricular decision making, and approach to and use of assessment. 

Organizational Structure 

We had originally hypothesized that the district operated within a top-down decision-making model. 
However, some of the information we gathered from both the observations and interviews suggested the 
possibility of a bottom-up model. Three areas were of particular interest: 

1. At Beta II, there was a strong commitment to decisions teachers had made 
collaboratively with a former principal. 

2. Teachers, principals, and administrators discussed "site-based management" 
approaches to decision making. 

3. Mechanisms, such as the district's Teacher Advisory Committee and the 
Building Advisory Committees, provided a means for teacher input. 

In our analysis we took a closer look at these three areas to determine their consistency with our 
original hypothesis. In this section, we discuss the evidence we found for both the top-down and the 
bottom-up decision-making models, and discuss the model that we believe fits the decision-making 
process in Beta. 

Evidence for Top-Down Model 

The comments of most of the participants and the behavior of two of the teachers strongly supported 
a top-down model. Discussions of teachers' input into curricular decisions confirmed the model and 
helped us to determine the flow of communication and the degree of control and influence exercised 
by different participants. 

Comments Made During Interviews 

When discussing decision making in the district, teachers, principals, and administrators explicitly and 
repeatedly stated that the central administration exercised ultimate executive control. When asked about 
curricular decisions, Mr. Sawyer, an assistant superintendent, explained: 

What you would see is the administrative hierarchy as far as organization. . . . Then 
you just feed the committees underneath. ... 

Mr. Sawyer, 4/3/89 

Mr. Dixon was quite straightforward in his view of administrative control: 

Right now I think it's a kind of lop-down process, with the superintendent being pretty 
much in charge. I've always felt a lot of leeway to make some decisions. With the 
current administration that's been curtailed some. . . . 

Mr. Dixon, 3/8/89 
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Ms. Carter's views are typical of the teachers interviewed: 

A lot of teachers feel as if they aren't giv[en] enough input. It sounds as if they are, 
but a lot of teachers feel that they give the input but the administrators make the 
decision. 



Choice of words and phrases within the interviews also helped to confirm our hypothesis of a top-down 
decision-making model. Teachers and principals often used words such as allowed and supposed to. 
This implied that the decisions they referred to were made by others, not by the individual teachers. 
For example, when asked about the role of the curriculum guide, Ms. Carter stated: 

We are supposed to follow it and everyone does the best they can. For example, we 
are supposed to teach so many units. 



While Ms. Boyd discussed the prospect of becoming an Accelerated School, she talked about the district 
allowing Beta II to deviate from the curriculum. 

We have been chosen now as an Accelerated School. The district has allowed us to 
do some things that they may not allow another school to do. In other words we can 
deviate from the curriculum. 



Another telling piece of linguistic evidence was the frequent distinction between "we" and "they" in 
discussing decision making. The use of "they" when referring to those who made final decisions as 
opposed to "we" made it difficult to determine specifically who held the responsibility for decision 
making in Beta. However, the "they" so frequently referred to seemed to be the central office 
administrators, headed by the Superintendent. For example, Mr. Dixon reported that he would 
implement his own decisions until someone told him not to. 

I never ask anybody if I can do something. I've done this long enough that I've 
learned if you ask somebody and they tell you, then you've got to do it. If you just do 
it, you're going to be able to do what you want to do for a long time before they even 
know what you're doing, if it isn't what they want. 



Behaviors in Interviews 

In a few instances, teacher behavior revealed an uneasiness with Ihc interview process. For example, 
one teacher covered the tape recorder during the interview. The interviewer had asked if teachers could 
use only one of the two district-adopted textbooks with the whole class. When the teacher replied thai 
she didn't "quite get that feeling," the researcher asked where she got those feelings. She laughed, 
covered the tape recorder, and responded. Another teacher asked to have the tape recorder turned off 
for the interview and expressed concern about notes being taken during the observation. Later, when 
the researcher asked this teacher how she became involved in the research project, she replied that the 
principal asked her to participate in the project, and added that given a choice, she probably would not 
have volunteered. 



Ms. Carter, 3/3/89 



Ms. Carter, 3/3/89 



Ms. Boyd, 3/15/89 



Mr. Dixon, 3/8/89 
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Interestingly, teacher remarks made during these interviews did not seem controversial, inflammatory, 
or even critical of the administration. Yet teachers were clearly uncomfortable with the project. Two 
explanations seem equally plausible. First, they may have been uncomfortable with universily-based 
observers in their classrooms. Second, they may have resented the process by which they were 
"volunteered" for the project. These behaviors suggested that the teachers were not integrally involved 
in the decision-making process. Had the teachers be~.n more involved in the decision, they could have 
freely chosen whether to participate; in that situation, they might have been less concerned about their 
remarks. 

Teacher Input 

In many of the interviews, people talked about administrators actively seeking teacher input, curriculum 
committees making curricular decisions, and administrators making final decisions. To determine the 
relationship between teacher input and the final administrative decisions, we examined the teachers' role 
in the curricular decisions. We learned that teachers provided input for district-wide curricular decisions 
(such as the selection of a new textbook series) and that teachers' feedback did not necessarily influence 
the curricular decisions. 

Two separate examples illustrate this relationship. In the first situation, kindergarten and first-grade 
teachers were upset when the district unilaterally decided to put Writing to Read labs in every elementary 
school. This was a district-wide decision that provided structured instruction of reading and writing for 
all kindergarten and first-grade students in the district. Mr. Dixon described the decision process in this 
way: 

That did not go through the reading committee. There was no real discussion on 
anybody's part outside of the Superintendent] and the Assistant Superintendent] and 
I think maybe the Directors 5f Elementary Curriculum. . . . And the general feeling 
in the staff was that they didn't like that they arrived without having anything to do 
with it. The curriculum committee had nothing to do with it. 

Mr. Dixon, 3/8/89 

Mr. Dixon added that after teachers saw the positive effects on students who had received a year's 
experience in the lab, they accepted the Writing to Read labs. The situation, he thought, demonstrated 
the open-mindedness of the teachers. He also noted that some teachers in the district had not yet 
accepted the labs. 

A second example centered on the purchase of science "tubs." Apparently, years ago, the district 
purchased science tubs which were stocked with equipment and manipulatives needed for hands-on 
science instruction. They were circulated around the district. However, materials needed for instruction 
were often missing from the tubs. At other times the tubs did not arrive on schedule. If they arrived 
the very day the teacher needed the tub, teachers would not have time to replace missing materials. 
Teachers, therefore, voted to adopt science textbooks in place of the science tubs, and the administration 
gave the tubs to another district. Last year, surveys were sent to the teachers for input on the science 
curriculum. The teachers asked for manipulatives that would supplement their science textbooks. Then 
"somebody" at the district level decided that the old science tubs would be used in the classroom. 
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Well they say that's what the teachers wanted. . .but that's not what we wanted. Now 
the way they are trying to pacify people, they are saying that they won't take our 
science books away. We can still use the science books but that we are going to use 
the tubs to teach three units. 

Ms. Carter, 3/3/89 

At grade-level meetings, some teachers expressed frustrations with these decisions. 

Our grade level was kind of calm. Third-grade teachers seem to accept things whether 
we agree or not. We just kind of flow. All we worry about is the testing. But the 
other teachers said their grade-level meetings were almost like shouting matches. All 
they did was talk about the tubs. 

Ms. Carter, 3/3/89 

Although teachers believed that decisions for a Writing to Read lab and for science tubs were made 
without-or in spite of~teacher input, the principals believed that teachers had input into curricular 
decisions. Mr. Dixon, principal of Beta I, joined Ms. Carter in her first interview. During this interview 
he stated that he "personally [thought] that teachers [had] a lot of opportunity" for input into curriculum. 
Indeed, Mr. Sandstrom, the principal from Beta II, believed that teachers had more input than principals 
in the curriculum. 

I would suggest that the principals feel that the teachers are heard far more than we 
are by the central office. We feel a lot of times disenfranchised from the decision- 
making process. We are informed as an aside. 

Mr. Sandstrom, 3/16/89 

There are lots of different levels of [instructional decision making]. That's a very 
complex thing in this district, because there is a lot of teacher input into how the 
decisions a r e made. 

Mr. Sandstrom, 3/16/89 

Although administrators and principals believed that teachers had input in curricular decisions, these 
two examples of curricular decisions in Beta suggest that while teachers might offer input into 
M centralized M decision making, they have few officially sanctioned opportunities to make their own 
curricular decisions at the classroom level. Teachers seemed to "teach behind closed doors" when 
implementing instruction and materials that might deviate from the district materials. As Mr. Sandstrom 
explained: 

Obviously what happens 99% of the time is that teachers do what they want until 
caught or whatever. Which is standard operational procedure all over the country. 

Mr. Sandstrom, 3/16/89 
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Evidence for Bottom-Up Model 
Past School-Based Decisions in Beta II 



During the interviews, teachers and administrators frequently discussed decisions the Beta II staff had 
made about eight years ago. Beta II had had special problems that needed to be addressed (e.g., low 
test scores, poor attendance, and behavior problems), and the former principal led the teachers in a 
rather lengthy set of deliberations to determine how to change the situation. A school-wide star chart 
with every child's name was placed prominently in the main hallway to acknowledge publicly individual 
progress. Rules for behavior were posted in each classroom. Programs were developed to increase the 
interest and involvement of parents in their children's school achievement. The staff decided upon a 
direct instruction reading and math series as a means of increasing students' standardized test scores. 
The former principal, Ms. Gillette, had asked for and received from the central administration 
permission to deviate from the district adopted textbooks by using these direct instruction materials. 

In our interviews, we sensed a high regard and affection for Ms. Gillette. Indeed, an Excellence in 
Education sign outside the school had been dedicated to her. However, teachers emphasized that Ms. 
Gillette's approach to school problems was unique within the district. They noted that while decisions 
had been made at the school, the teachers and the principal did not have the freedom to implement 
their decisions without the approval of the administration and board. Rather than evidence for a 
bottom-up organizational structure, the decisions made at Beta II became evidence for how exceptions 
were handled within a top-down structure. 

School-Based Management 

As we began the cycle of the interviews and observations, district-wide school-based management was 
in the planning stage. Beta administrators and principals were discussing and negotiating what 
restructuring might mean in Beta. Beta I and Beta II had been selected by the district as the only two 
Beta schools which would participate in the initial restructuring efforts. 

Beta I school had applied to become part of this school-based management program. Teachers and the 
principal talked of it as a possibility for implementation the following fall, pending confirmation of 
funding. When discussing this possibility, the principal contrasted it with the current top-down decision- 
making model: 

Right now, I think it's a kind of top-down process, with the superintendent being pretty 
much in charge. I've always felt a lot of leeway to make some decisions. With the 
current administration that's been curtailed some, yet I find them promoting us to be 
part of the restructuring activity. I think there's a little bit of conflict there, and 
perhaps the decision-making process will be closer to the school and where the action 
is as we move through this process than I think it is now. . . . [Indeed] we may be 
beyond that [top-down decision-making] now, particularly since they're suggesting, 
people in the central office, that we become involved with restructuring. 



Mr. Dixon, 3/8/89 



Beta I currently had a Building Advisory Committec-a committee of teachers who advised Mr. Dixon. 
In the interviews, teachers at Beta I acknowledged that while they currently were involved through this 
committee, there would be major changes in the decision-making structure if the school became involved 
in the restructuring project. For example, at the time of the data collection, teachers at Beta 1 were 
allowed to make some budget decisions (e.g., teachers were given a discretionary budget of $100 for 
materials). The restructuring process could potentially give teachers freedom to make major budgetary 
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decisions decisions they believed were not permitted in other schools in the district. One teacher 
expressed uncertainty as to how much control the teachers would actually have: 

It's asking a lot, and I'm sure we won't be given all the freedom that we're seeking. 
(She added, "And it's probably best we're not.") 

Ms. Holmes, 3/7/89 

School-based management was also discussed by Beta II teachers. In the Beta II interviews, they talked 
ot the restructuring process of decision-making" and of "accelerated schools." Central office 
administrators had nominated Beta II to become an accelerated school. According to the principal of 
Beta I, the school had been "designated [as] an accelerated school for at-risk kids." As an accelerated 
school, the principal and teachers would be allowed to make decisions normally reserved for the central 
office, to deviate from the district curriculum, and to develop programs and curriculum especially for 
the students in their school. The principal would lead the teachers in decision making. Mr. Sandstrom 
was unsure how this shared decision making would work. 

[The central office] wondered if I would be willing to accept the restructuring process 
of decision making. Whether I can handle and monitor it so it doesn't get out of hand. 
. . . It's somewhat of a difficult framework to lead because we are so used to being 
told what we're to do. So we are still struggling with what that means, particularly at 
this moment, with how far does that go. 

• Now if we are accepted as an Accelerated School we're supposed to have more latitude 
but I don't know that I necessarily sense that latitude yet. 

Mr. Sandstrom, 3/16/89 

When asked if, as part of an accelerated school, he still would have to get approval for everything from 
the central office, he replied: "We're still negotiating what that's going to involve." 

The restructuring process in the early formative stage seemed to affect the way people talked about 
decision making and may have been responsible for some conflicts and inconsistencies noted in the 
interviews. According to Ms. Eaton, teachers were aware that the administration was "trying to include 
teachers in the decision-making process." In reference to instructional decisions in the classroom Ms 
Boyd observed, 

I think the district is good at rreating us as if we know what we're doing. In other 
words, they don't make the ultimate decision. 

Ms. Boyd, 3/15/89 

However, in the first interview, she had made this statement: 

Teachers serve on these committees and then we make the decisions. We think we 
make the decisions. Then, people in charge make the ultimate decisions on a lot of 
things. 



Ms. Boyd, 3/8/89 

By viewing these quotes within the context their recent entry into the network of accelerated schools 
perhaps Ms. Boyd's contradictory remarks can be viewed as simply reflecting the initial stage of the 
restructuring process. Teachers were aware that shared decision making was likely to become a reality 



22 



Assessment and Decision Making in Beta - 19 



however, they also understood that the decision-making control was with the administration, even the 
decision to share control with teachers. Principals and administrators were in the process of determining 
what role the teachers would have in decision making. In reference to the restructuring project, an 
Assistant Superintendent stated: 

That speaks to teachers making more of these local decisions. We're not sure what 
those local decisions are, but it should be more than the purchase of toilet paper. 

Mr. Sawyer, 4/3/89 

Mr. Sandstrom, the Beta II principal, seemed to agree as he noted it was "hard to get a staff who is used 
to autocratic ruling used to shared decision making." He added: 

I found that this staff is naive about their responsibilities and about how they could 
make decisions and what they had to say was important over and above whether or not 
there is toilet paper in. the bathroom. 

I've always recognized the expertise that teachers have, but these teachers are just not 
used to having to make these decisions. They are not really good at or sophisticated 
at making global kinds of decisions. It's hard for them. 

Mr. Sandstrom, 3/16/89 

Principals seemed to be uncertain about their role in a restructured school: How much latitude would 
be allowed them? How were they to monitor the process so "it doesn't get out of hand?" How could 
a school-based decision-making process be imposed on a district whose members were accustomed to 
others making those decisions? 

Mechanisms in Place for Bottom-Up Model 

In spite of their persistent view of Beta as a top-down district, principals and administrators noted that 
teachers in Beta were given more opportunities for input than they might have in other districts. 

Now if you were to go to another district and take this process we have on the road, 
there will be s^me districts that say, "Your staff has an awful lot of input." 

Mr. Sawyer, 4/3/89 

Frequently they referred to committees as the place where teachers participated in those decisions. 

I would say that our teachers are used to being involved in making decisions. They see 
K-12 [committees] as giving them that opportunity. Now we do have teachers who 
want to make more decisions closer to classroom. 

Mr. Sawyer, 4/3/89 

Mr. Sandstrom noted that more teachers were involved in the decision-making process in this district 
than in other districts. When asked what he attributed that to, he replied: 

I would attribute it to the central office not wanting to dictatoriaily impose their 
decisions. . . . It's been fluctuating according to the degree to which the central office 
wants to decentralize decision making. It happens now that the central office is 
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Mr Sandstrom, 3/16/89 

met wuh the superintendent once a month to communicate concerns to the suirintenden, Mr 
Sandstrom saw the posstbdit, that curricular issues could be discussed mom in .^ ,1* 

always agree upon, advisory in nature. 

Mr. Sawyer, 4/3/89 

The BAC and the TAC seemed to be committees where the concerns of teachers could be aired but 
were not vehicles for decision making. They provided the means for communStit with fhe 
administration and a forum for teachers to discuss issues. communication with the 

Discussion 

While the decisions made at Beta I, the talk about school-based management and the committee 
structure appeared to point to a bottom-up decision-making model, closef anal^is rev a e d t^aT he 
did, in fact, support our original hypothesis of a top-down model. The decisions ma^e at Beta T were 

52K£S3 Ld» Ce 7 al 3dminiStrati0n ; SCh °°'- b - d management rtrbe montredlo k 
tot£^ ^ W - 1 illustrates the 

[Insert Figure 1 about here.] 

Curricular Decision Making 

Within this top-down administrative structure, decisions about curriculum became decisions about 
materials. Teachers provided input into curricular decisions by completing needs assessmenufor Teh 
subject area up for textbook adoption and by serving on the curriculum comST Th needs 
assessment forms provided a means for teachers ,o evaluate the current curriculum and materials 
Before filhng out the form, teachers examined portions of the district's curriculum the s^e learnmt 
objec ives and approbate documents such as the Standards of the National C^of£££3 
Mathematics Based on that information, teachers made suggestions about what should be added or 

£ obtt C ' CVC i ThCy r tCd dCgrCC t0 Whkh opcrationalized cu r culum a tuaHy m 

ts objectives. The second part of the Needs Assessment contained questions that permitted teacher 
to identify materials they wanted to use and to evaluate the strengths and weaknesses o he p sen 
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materials and program. The needs assessments were then given to the appropriate curriculum 
committee. 

The curriculum committees mentioned in the interviews were reading, language arts, math, science, 
social studies, health, and substance abuse. The committees were comprised of teachers who, as a 
group, represented all grade levels and all buildings. The administration assigned the grade level to be 
represented by each building on the committees and the principals communicated to the teachers what 
committees needed teachers. Teachers then would either volunteer or be asked by the principal to serve 
on these committees. Some principals also served on these committees. Mr. Sawyer, an assistant 
superintendent, chaired some of the curriculum committees and as Dr. Montgomery, the Superintendent, 
noted: 

His directors chair the remainder. They all flow through his vision in the schools. 

Dr. Montgomery, 4/3/89 

This 20 or more member group formed a district-wide committee that determined the district-wide 
curriculum and materials. 

The curriculum committees determined the objectives for each grade level and decided on textbooks 
for district-wide use. The textbook adoption process operated on a six-year cycle. In the first year of 
the cycle, the curriculum committee decided on three or four sets of materials for piloting. During the 
second year of the curriculum adoption cycle, teachers were selected to pilot the materials and to 
provide the committee with monthly feedback on "how that curriculum works." If there were any "gaps," 
then supplemental materials would be found. The next four years of the cycle were for monitoring and 
assessing the district's curriculum materials. 

When describing the committee's role in the adoption process, Ms. Warner, an Assistant Superintendent, 
observed: 

In the initial [committee] meeting we set the agenda for the year. As we are in the 
process of looking at materials in the language arts, we need to set up what we expect 
to have for the next year. 

Ms. Warner, 4/3/89 

Then, when she was asked about goal setting, she responded: 

We view the objectives. Teachers will do a needs assessment, evaluation of current 
curriculum. In light of the learning assessment plans they would look at the objectives 
and look at our current material in the curriculum. 

Ms. Warner, 4/3/89 

Ms. Boyd expressed the role of the teachers on the committee in the final curricular vision in this way: 

Teachers serve on these committees, and then we make the decision. We think we 
make the decision. Then, people in charge of the curriculum make the ultimate 
decision on a lot of things. 
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The committee decides on the materials or the length of time that well be teaching 
it. The committees buy and decide what series we will use. 

Ms. Boyd 3/8/89 

Through this process, the district had adopted two reading series for district-wide use. Except for the 
teachers in Beta II, teachers were expected to use these two reading series. Teachers at Beta II were 
expected to use the reading series approved by the district for their school. One administrator stated 
it this way: 

We are restricted in eight of our schools to [two reading series]. . . . You can select 
between them. ... At Beta II we are restricted to [a basal series] and [a direct 
instruction approach]. 

Mr. Sawyer, 4/3/89 

Teachers also talked about this expectation. 

There are two series chosen; and we are told that we have a dual basal system. You 
get the feeling that you are expected to use both. . .both with the class, not with every 
child. You use one or the other. 

Ms. Holmes, 3/7/89 

I have certain books I have to use. 

Ms. Eaton, 3/9/89 

How they used them and what would be emphasized were determined by the individual teachers. 

Administrators believed that by adopting two reading series, teachers had the flexibility they needed to 
provide appropriate reading instruction. One series was to be used with more skilled readers and the 
second series with the other children. However, some teachers seemed to feel a need for additional 
options. 

Ms. Eaton had moved from a district that used a literature based reading program to Beta II, with its 
direct instruction reading program. She saw that the Beta school reading series was so structured that 
it affected her instructional decisions. After explaining that her philosophy of reading differed from the 
direct instruction reading program, she described the inflexibility of the series: 

That reading group time is very structured and it is set. The decisions are made. The 
script is there and it is so structured. 

Later she stated: 

I think it's important, when you arc analyzing my decisions, lhat you know where I'm 
coming from. 

Ms. Eaton, 3/9/89 

Although she used the direct instruction reading scries, she also added to her program the elements of 
reading and writing that she viewed as important. She used the low scores of district-wide standardized 
tests to support her decisions. 

r 1 
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Beta's process for curricular decision making is summarized in Figure 2. 



[Insert Figure 2 about here.] 



District-Wide Assessment 



Assessment-as-Tests 



The administrators, principals, and teachers who participated in this study seemed to define "assessment" 
as tests selected for district-wide use. One teacher, for example, when asked how she assessed students 
in the broad sense, replied: 

The biggest thing would be tests. The test is the big thing. 



The test score data from published tests (e.g., the SRA and the state reading and math assessments) 
were reported to the public in the state mandated School Report Card as a means of holding districts, 
schools, and teachers accountable for student performance. Teachers modified their instruction based 
on the outcomes of these tests and were concerned about covering the material that was on the tests. 

Mastery of the district objectives was tested each spring. Teachers reported that students in grades one 
through five were given the SRA test, and the third grade was also given the state reading and math 
tests. One teacher mentioned giving a group IQ test. Additionally, district tests were developed to 
cover the objectives not covered by the standardized test and state tests. One such test was a writing 
sample assessment teachers were required to give for all grade levels. 

The district did not require teachers to use the textbook's published unit and chapter tests, although 
many teachers did choose to use them. In Beta II, the teachers used the tests from the direct instruction 
reading series after every 20 lessons. The teachers in Beta I were "given the opportunity" to use the 
published tests from the two district-selected reading series. Teachers in both schools reported using 
the reading series tests at the beginning of the school year to help determine ability groups. 

The educators we talked to in Beta seemed to believe that tests covered what was important for children 
to learn. Administrators and principals seemed to view published tests as a means to obtain objective 
information that would help them evaluate the quality of instructional programs the children were 
receiving. Mr. Sandstrom expressed the value of published tests in this way: 

It's less intuitive and more objective. . . . There is more objective data. Along with 
this, there is the teacher's opinion, which is valuable. For example she might recognize 
that a child did poorly on the test because he or she was having a bad day. . . . The 
danger in abandoning the formalized measure is that often I hear teacher's assessments 
of students and they are incorrect. 



Most often test scores were questioned if the teacher observed the child having a "bad day." 
Administrators and principals mentioned teacher observations of students' test-taking behavior rather 
than of students' learning and classroom performance. It was felt that teachers could shed more light 
on how the students took the test rather than providing additional information about the student's ability 
based on her observations or other informal assessments. Published tests provided the objective 
information administrators needed for the State Report Card. 



Ms. Boyd, 3/15/89 



Mr. Sandstrom, 3/16/89 
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Principals were required to compile the test scores for their buildings and to send them to the district 
administration. The administrators used the tests scores to inform them of the progress the students, 
as a group, were making within each school building and district-wide. 

Standardized achievement tests and state tests also provided indices that could be reported to the public. 
Mr. Dixon summarized the relationship of assessment and accountability in this way: 

It occurs to me that since we started doing that [sending the state report card to every 
family] teachers are more concerned about what the test results are, because those test 
results now go into everybody's home. I know we pay much more attention to them; 
and I know that in some things in the curriculum we are more careful about getting 
them taught because we know they are going to be evaluated. In fact, we are very 
careful now, since we went to the spring testing, to see that everything that's on the 
test that's normally in the grade-level curriculum is taught before the test is 
administered. 

Mr. Dixon, 3/8/89 

Teachers were concerned about the reporting of test data, as expressed by Ms. Carter: 

Well, we do worry. The results get published and we want the school to look good. 

Ms. Carter, 5/10/89 

Another teacher expressed concern about the public's interpretation of these standardized test scores. 

I think [testing] drives the whole system more than it should. . .because test scores are 
published in the paper by school, and it's right there. I guess that it goes along with 
the feeling that the school must be accountable and that parents need to know, but I 
guess that I feel tests can be misinterpreted, too. 

Ms. Eaton, 3/9/89 

These data suggest that, in Beta, the standardized test results in the State Report Cards was viewed as 
a means for holding teachers accountable for implementing the district curriculum: Teachers were 
expected to cover what the students would need to know for the tests. Teachers expressed concern 
about the effect of such accountability. Ms. Eaton saw that assessment, used for accountability to teach 
a district curriculum, affected the individual reading instruction she could provide: 

[A]nd now we're back toward assessment and accountability and district wide objectives 
in reading rather than an individualized program. 

Ms. Eaton, 3/9/89 

She believed that, as one was held accountable to district-wide objectives through formal assessments, 
the reading program became less individualized. Movement toward the structured formal assessment 
diminished possibilities for individualized reading programs. To cover the material before spring testing, 
teachers felt that needs of individual students were not met. Ms. Carter stated it this way: 
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There was just so much to cover before the test. We don't think it's right that the kids 
get tested on what they haven't been taught, and so we try to cover everything before 
the test. It's really a push. Kids get left behind. Now [after testing is over] we can 
go back and take the time and help the kids that got left behind. 

Ms. Carter, 5/10/89 

The Influence of Assessment-as-Tests on Instruction 

Use of tests directly affected instructional decisions in the district. First, teachers used instructional time 
to prepare students for tests. During one observation, for example, Ms. Carter was preparing her third 
graders for the district writing assessment. On the board was written the following writing prompt: 

Your principal has said that wearing shorts to school will not be allowed because some 
students wear them when it's too cold, some wear raggedy cutoffs, and it gives students 
the attitude that they come to school to play instead of to learn. Agree or disagree. 
Explain. 

Field notes, Ms. Carter's Classroom, 4/28/89 

After students wrote their opinions, Ms. Carter checked their work. She would say things such as H good 
reasons, good ideas, okay good now give me another reason, reread this. . .these aren't all complete 
sentences, okay good, now you need a closing sentence." She explained that because of the test she 
worked on writing all year and wanted the children to le^irn to write using a topic sentence, three or four 
sentences and then a closing sentence. 

A second way tests affected instructional decision making is that test scores were used to inform 
teachers, principals, and administrators of instructional areas needing modification. 

[The] principal would talk to us if scores were low. . .other than that, we look to see 
if we could have served them better if we had changed something. . .make 
modifications in our plans for teaching. . .to improve their deficits. 

Ms. Carter, 3/3/89 

Indeed, teachers changed their instruction if standardized test scores for a group of students were low 
in a specific area. As Mr. Dixon explained: 

[The unit leaders and I] look at the [test results] together and see what the strengths 
and weaknesses are and point that out, and then teachers use that to help themselves 
develop directions to try to meet those things. 

I would say to [a] teacher, "Take a look at what you're doing with nouns and pronouns 
and recognize that that's an area your kids didn't do as well in, and you might want to 
spend a little more time or effort or even rethink. . .what you are 'loing." 

Mr. Dixon, 3/8/89 

Another example of modifying classroom instruction on the basis of tests scores was provided by Ms. 
Cooper, Director of Secondary Curriculum: 
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We found that our students scored very low in antonyms, and what we've decided as 
a department was that our goal is going to be that we'd emphasize the instruction of 
antonyms for our kids. 

The next spring the student scores were up. She reported: 

[It was a] positive reinforcement to the teachers because they thought, "Oh those five 
minutes I spent on antonyms paid off. M It's not like they went through any major 
lesson plan changes, but they just did some modification of instruction. 

Ms. Cooper, 4/3/89 

Ms. Eaton felt that focusing on comprehension was an important part of reading. She reported: 

[SRA test scores] indicate that our children need more work with comprehension, 
higher-order thinking skills, predicting. 

Ms. Eaton, 3/9/89 

She used this information to justify augmenting her direct instruction program with more writing and 
comprehension instruction. 

Our test scores support my rationale and beliefs as to what needs to be added to the 
program. 

Ms. Eaton, 3/9/89 

Spring testing also seemed to influence Ms. Holmes' instructional decisions. She felt an urgency to 
make sure the students learned what they needed to know to take the test. To get the students ready 
for the SRA tests, it was necessary to disregard the sequence of the textbook. However, the textbook 
was structured to provide reinforcement of previously taught concepts. Because of this structure, she 
found it difficult to skip around in the math book to cover the concepts that would be tested in the 
spring. Her decisions of what pages to use were influenced by what was on the spring tests. 

I'm at the point now where I want to do as much as I can, let them learn as much as 
they can before the test. 

Since I've seen the test, I've done my long-range planning differently. I would like to 
expose them to all areas of math. For example rather than waiting until we get to 
telling time, when we get to that chapter, also our math book is such you can't really 
skip around. It does a check up at the end of each chapter on the previous chapters 
which I think is good for the review. 

Ms. Holmes, 3/30/89 

Tests seemed to be used to identify those instructional areas teachers could modify for students as a 
group rather than for an individual student. 

The third way testing influenced decisions was through its impact on curriculum/materials, as the 
following remarks illustrated: 

Ms. Eaton: "I think they arc really trying to move towards a new math program for 
example." 
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Interviewer: "What is driving this change?" 

Ms, Eaton: National assessment in math, the National Council of Teachers in Math, 
their new proposals, these are the requirements that we would like to see in 
math. . . . We're supposed to fill out the math [needs assessment] based upon the new 
recommendations. 

Ms. Eaton, 3/9/89 

When another teacher was asked how tests affected teachers,.she responded: 

Well, materials. We adapt materials to fit the test. Like the state test says, there is 
more than one right answer. Well, it may not be right, but we teach that there is just 
one answer, so now we have to change the materials to fit the format. 

Later she stated: 

Tests will change texts and then the teaching would match the test. 

Ms. Carter, 5/10/89 

Informal Assessment 

Administrators, principals, and teachers all seemed to equate assessment with tests, but to have differing 
perceptions of what constituted "informal" assessment. The administration seemed to consider informal 
assessment as the nonstandardized textbook tests given by teachers. For example, when asked about 
informal assessment, Ms. Warner replied: 

I think most of what you see is that, since you have to give some kind of progress 
report after six weeks, that there is some type of assessment, be it a unit test or a 
chapter test or something. 

Ms. Warner, 4/3/89 

Principals and teachers considered informal assessment as unit and chapter tests as well as part of day- 
to-day decisions associated with instruction and materials. Teachers, for example, referred to the 
questions they asked children as informal assessment. How the children responded to the questions 
helped teachers determine if the students understood the skills and/or concepts in the materials. 
Teachers in Beta II emphasized that reading assessment occurred daily because it was structured into 
the direct instruction program. 

With direct instruction, it's really a constant assessment. It's ongoing. It's day after 
day after day. And that's how these kids never fail. 

Ms. Boyd, Teacher, 3/15/89 

Ms. Eaton had an especially broad view of informal assessment. For her, "informal" assessment included 
the reading series tests, questions for checking comprehension, writing samples, and knowing the 
children and what "they arc capable of doing and how to get Ihem to reach their maximum potential." 
For this teacher, informal assessment depended on the teacher's perceptions and observations of 
individual students; and she used them to monitor the development of each child's learning. 
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During their interviews, three of the four teachers discussed some type of informal assessment. These 
informal assessments-observations, samples, checklists, narratives, and notes-were closely tied to 
mastery of materials and/or test preparation. 

Of the three teachers who discussed informal assessment, two used informal techniques for placing the 
students into ability groups, checking skill progress, and monitoring worksheet accuracy. There was 
some evidence that teachers might skip a workbook page if they knew that students understood that 
skill. 

Informal notes; if I find a child is having trouble, Til write it down. I have seven LD 
children and if they are having trouble with the story or with a certain part of the story 
or with a skill, Pll tell the LD teacher that day. 

Ms. Carter, 3/3/89 

For these two teachers, informal assessment provided information about how the students were 
progressing within the curriculum materials. 

An example of informal assessment as a placement tool was given by Ms. Holmes, a first-grade teacher. 
She used the screening results, kindergarten teacher's comments, and her own observations and informal 
assessments to determine reading groups. She would ask students to identify colors, letters, numerals, 
color and number words, and some vocabulary words. She observed what they would do when 
encountering unknown words-did they try to sound the word out or just read on? She required students 
to complete activities in a readiness book to determine "who can sit there, go left to right, [and] follow 
along with me." She assessed the children to see if they could identify which pic are did not belong in 
a row. Ms. Holmes observed students to determine "who can stay with me, who picks up on things 
easily, who's going to need more repetition, [and] more help." If this initial assessment was not Tine 
enough" for a particular student, she would then give the basal series placement test, which helped her 
make some decisions. Ms. Holmes used several pieces of "assessment data" to determine groups: what 
other teachers said about a student, her own informal assessment, and possibly a published test. 
Informal and formal assessments were used for placing students into reading ability groups by 
determining what skills students had and at which pace they could go through the material. 

Ms. Eaton used informal assessment to monitor the development of the students' writing. She kept a 
writing folder for each child, made notes about the children's writing, in a notebook, and held 
conferences with the children about their writing. She used informal assessment to monitor individual 
students' progress. She accomplished this by knowing each child and what "they [were] capable of doing 
and how to get them to reach their maximum potential." 

Teachers talked a great deal about such informal assessment. Indeed, comments about samples, 
checklists, and dynamic and other forms of informal assessment accounted for 43% of all teacher talk 
about assessment (see Figure 3). Teachers also reported relying on informal assessments to make a 
number of decisions at the classroom level (see Figure 4). Assessment-as-test and district-mandated 
materials (which teachers discussed the other 57% of ihe time) often provided the criteria for such 
informal assessments. For example, teachers informally determined if a student got the questions right 
on end-of-chapter tests or basal worksheets or if their reading and writing skills were consistent with 
what would be measured on district, state, or national tests. In this way, responsibility for determining 
"what matters" instructional^ seemed to lie outside the classroom-with state mandates, tests, and 
textbooks. Teachers made decisions within the parameters established by these outside sources. 



[Insert Figures 3 and 4 about here.] 
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Conclusion 



As we were in the process of analyzing our information from Beta, we found ourselves searching for an 
overriding metaphor to explain how the district operated. Members of the research team suggested a 
variety of possibilities: a factory, a traditional school, an old style business management model, and the 
military. Various aspects of Beta could be described using these seemingly different "pictures," yet each 
of them shared with Beta these similar themes or aspects: 

- The power resided at the top of the structure. 

- Those at the bottom of the power structure were implementors of decisions. 

- Decisions reflected the attempt to ensure continuity throughout the system. 

- Indices of the system's success were based upon group performance. 

Additionally, we found that decision making in Beta seemed to be driven by accountability to the public. 
Inherent in this was the belief that standardized tests provided an objective, efficient, and appropriate 
means of evaluating educational programs. Beta had accepted that accountability was inevitable. 
Objective measurements were valued by administrators and principals, and teachers felt that they were 
held accountable for their student's performance on these tests. In addition, teachers were to implement 
instruction that would provide students with the tools to enable them to do well on the tests. 

In many ways, Beta typified what we commonly consider to be "school." The physical, organizational, 
and instructional aspects of Beta were similar to many of the schools with which the researchers were 
familiar as students, teachers, and researchers. Traditional roles were assumed by members of Beta: 
teachers taught; each principal was responsible for coordinating the activities that represented school 
in his building; and the administration made final decisions before taking them to the board for 
approval. District-wide curriculum and assessment were used to ensure quality education. It was our 
sense that much of what was said of Beta would typify many schools. 
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"A lot of teachers feel as if 
they aren't giving enough 
input. It sounds as if they 
arc, but a lot of teachers feel 
that they give the input but the 
administrators make the decisions.' 

(Teacher) 
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CURRICULUM DECISION - MAKING PROCESS 



"THEY 




ERIC 



3S 



Figure 3. BETA 
Approximate amounts of teacher talk, by type of 
assessment (in lines) 








%of Talk 
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149 lines 
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26 lines 
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District 


0 lines 
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260 lines 
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Teacher Made 


0 lines 
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330 lines 
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Figured Uses of Assessment: Teachers' Perspectives 
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Center for the Study of Reading 



174 Children's Research Center 
51 Gerty Drive 
Champaign, IL 61820 
USA 



217 333-2552 
217 244-4501 fax 



An announcement about a proposed research 



project 



at the Center for the Study of Reading at 



the 



University of Illinois at Urbana-Champaign 



A group of us at the Center for the Study of Reading have recently been involved in research on assessment. In 
addition to our work with the state-wide assessment here in Illinois, we have also been studying the role of 
reading assessment in schools nationally; for example, we have recently completed an extensive national survey to 
try to belter understand the relationship between reading assessment and instruction in our nation's schools. 

As a follow up to that broad survey, we are interested in trying to understand the same assessment-instruction 
relationship from a much more "contextualized" perspective-by working with teachers, administrators, students, 
parents, and policy-makers to understand how that relationship works in "their" school(s). So we are designing a 
national study, with four to six districts in Illinois and a like number from outside the state. In each district we 
would like to select two schools and four classrooms (two per school) to work with more intensively. 

We would talk with administrators, school board members, parents, teachers, students and support personnel in 
order to understand decision making from a variety of perspectives. In addition, we'd like to spend a goodly part 
of each of 4-8 school days, over the next four months, observing two teachers in each school. Wc want to 
understand the kinds of decisions they make on a daily basis and how they use a variety of formal and informal 
assessment practices to make those decisions. So we would not only observe but also talk with them about those 
observations. 

Wc see this research as an opportunity for us to better understand classroom practice and teacher decision- 
making, particularly from the perspective of the data teachers use and how they use it to make those decisions. 
Wc sec this as an opportunity for the teachers, schools, and districts who get involved to better understand their 
own practices in the context of how other teachers, schools, and districts approach these same issues. Our intent 
is to create an environment in which everyone involved has something to learn and benefit from. Wc would like 
our cooperating teachers and other school personnel to participate as fully *ds they would like-perhaps meeting 
with us to discuss findings and, if they so desire, to work with us as we seek to share findings and insights with a 
broader audience. 

For those who will participate only in an interview, the time commitment is modest, perhaps an hour per person 
at the outside. For the two teachers per school with whom wc work, it is a different story. We realize that 4-8 
days of being observed and talking with observers is time-consuming and puts additional demands on 
professionals whose time is already highly taxed. We realize too that the invitation to become learners in a 
cooperative venture may seem more attractive to us than to busy professionals. We are hoping, though, that 
some teachers will see this as an opportunity and work with us to eliminate the research/practice division that so 
often hinders communication between university and public school educators. Frankly, we do not think we can 
ever really understand the assessment-instruction link without seeing it happen in classrooms and then having the 
opportunity to reflect on decisions we observed with the teachers who made them. 
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CSR Research Announcement: Page 2 



In exchange for this help, we can offer these incentives: 

• Each participating school will receive a narrative report of decision-making/assessment practices in 
their school. 

• Each participating school and district will receive a copy of our synthesis of the 8-12 sites with whom 
we work. 

• We will hold working sessions so that educators from both the university and the public school can 
work together to understand and share the implications of our case studies. 

• We will also secure tuition and fee for participating teachers. 

• The principal investigator is willing to volunteer to work with any school or district level curriculum 
or assessment planning committees to the degree that they would like his involvement. 

This is an exciting time to be involved in education. We think this assessment research offers an opportunity for 
school and university to work together and to make a difference in the lives of teachers, researchers and children. 

We hope you will accept our invitation to become involved. 

Contacts: 

P. David Pearson (217) 333-7628 

Diane Stephens (217) 244-8193 

Center for the Study of Reading 
51 Gerty Drive 
Champaign, IL 61820 
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Interview Codes 



Slot 1 




Slot 3 




Slot5 




Talkina About 


Source 


ParticiDation 


a. 


Self 


301 


Mandate 


51 


Mandatory 


b. 


Superintendent 


302 


Board of Ed. 


52 


Voluntary 


c. 


Assistant Super. 


303 


Superintendent 






d. 


Board Member 


304 


Principal 






e. 


Staff Devi, (person) 


305 


Colleague 






f. 


Consultant 


306 


Staff Development . 






g. 


Principal 


307 


Book 


Slot 6 




h. 


Teacher 


308 


Teacher Education 


Tvnes of Assess. 


i. 


Student 


309 


Personal Experience 






i. 


Parent 


310 


Experience as a student 


601 


National 


k. 


State 


311 


Teaching Experience 


602 


State 


I. 


District 


312 


Intuition 


603 


District 


m. 


Administration 


313 


Canl Identify 


604 


Text Publisher 


n. 


Staff Dev. (program) 


314 


State 


605 


Other Publisher 


0. 


Decision Making 


315 


Professional Meeting 


606 


Teacher Made 


p. 


Curriculum 


316 


Reflection 


607 


Samples 


q. 


Instruction 


317 


Source 


608 


Checklist 


r. 


Assessment 


318 


Asst. Super. 


609 


Informal 


s. 


Discipline 






610 


Dynamic 


t. 


Materials 










u. 


Classroom 










v. 


School 










w. 


Committees 


Slot 4 


Slot 7 




X. 


Town 


Control 


Usfis/Role of Assess. 


y 


PTA 










aa. 


Asst. Principle 


401 


Self 


701 


Accountability 


ab. 


Social Worfcer 


402 


Cooperative 


702 


Program Evaluation 


ac. 


Education 


403 


Committee 


703 


Teacher Evaluation 


ad. 


Budget 


404 


Teacher 


704 


Pupil Placement 


af. 


Salesman 


405 


Principal 


705 


Reporting Pupil Progress 






406 


School 


706 


Monitor Pupil Progress 






407 


District 


707 


Choosing Materials 






408 


State 


708 


Instructional Decisions 






409 


Student 


709 


Diagnosis 


Slot 2 


410 


Aide 






Type 




411 


Superintendent 










412 


Asst. Super. 






21 


Philosophy 


413 


PTA 






22 


Policy 


414 


Union 






23 


Practice 


415 


Ad 










416 


Board 










417 


Parent 







Observation Codes 



Slot 1 




Slot 2 


Slot 3 




Task Definition 


Grouping 


Content 


1 

i 


A ccflcemflnf 


201 Whole/T 


^n 1 


OOUlal OlUQIcS 


o 


benavior 


202 Small/T 


OU<i 


•ocience 




ivi at (ay ci i ic i u 








O 


uiassroom 


203 indiv/T 


oUo 


Math 
Main 




ACuiVTy 








.4 

4 


l 1 I O M M t ^ / 

rianningf 


204 Whole/NoT 


OU4 


Liieraiure 




bcneduie 








c 
o 


n on - ACau e mic 


205 Small/NoT 


one 
305 


Heading 






306 Indiv/NoT 


306 


Writing 








307 


Grammar 








308 


Spelling 








309 


Phonics 








310 


Vocabulary 








311 


Music 








312 


P.E. 








313 


Drama 








314 


Art 








315 


Other 








316 


Health 


Slot 4 


Slot 5 


Slot 6 


Materials 


Tvoe of Activity 


Tvpe of Assessment 


400 


None 


« 

fUse onlvwith Slot 1 #31 


(Use 


onlvwith Slot 1 n^) 


401 


Text 


501 Telling 


601 


National 


402 


Basal 


502 Tl/SR 


602 


State 


403 


Trade Book 


503 Scaffold 


603 


District 


404 


Workbook/sheet 


504 Discussion 


604 


Txt Publisher 


405 


Blank Paper 




605 


Other Publisher 


406 


Kit 




606 


Teacher made 


407 


Manipulative 




607 


Samples 


408 


Computer 




608 


Checklist 


^09 


Tape Recorder 




609 


Informal 


410 


Other Gadgets 




610 


Dynamic 


41 1 


Art Supplies 








412 


Chalkboard 








413 


Homemade Book 








414 


Reference Material 








415 


Test 








416 


Other 








417 


Film/Movie 
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